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Program Overview 


The MacPhail Center for Music Online School Partnership Program aims to increase access 
to high-quality music education to students in rural public schools in order to improve 
student interest and knowledge in music, and to improve or maintain positive school 
engagement and non-academic skills. The MacPhail Teaching Artists work collaboratively 
with music educators in schools to develop curriculum, implement the Online School 
Partnership, and work with students individually or in small groups on their music lessons 
via the internet. Teaching Artists teach classes at the partner schools remotely and in real- 
time through teleconferencing technology. Classes are typically anywhere from 30-60 
minutes in length, with a MacPhail Teaching Artist leading sessions on any of the following: 
current repertoire the students are working on in their ensemble; selecting and working 
on repertoire for a student to prepare for an upcoming audition; general tone, technique, 
maintenance of instrument, etc. All of the session content is based on what the partner 
educator would like the Teaching Artist to cover — the Teaching Artist then creates the 
session structure based on what needs to be worked on the most during the given session 
time period. 


The Online School Partnership has been providing lessons and working with schools outside 
the Twin Cities metro region since 2011. It currently has 38 Teaching Artists, partners 
with 29! schools, and serves 1,350 students. This evaluation, however, only focuses on five 
schools: Kerkhoven-Murdock-Sunburg High School, Willmar High School, MACCRAY 
High School in Clara City, Rush City High School, and Lincoln High School in Thief 
River Falls. Thirteen Teaching Artists from MacPhail were involved in the Online School 
Partnership at the five schools that were involved in this evaluation. Online School 
Partnership offered the following types of lessons in participating schools: 


™ Group lessons: These lessons are ongoing and built on method books, handouts, and 
sequential practice planning. The MacPhail Teaching Artist consults with the partnering 
music educator to create ongoing assignments that encourage musicianship and solid 
playing techniques for the group of students, with a focus on one particular instrument 
or voice type. Eighty group lessons were held for 141 students at the five schools 
involved in this evaluation.” 


The Online School Partnership worked with more than 44 schools in the 2015-16 school year, but only 
29 schools are a “partner” school, defined as those schools that participated in 10 or more online 
sessions with MacPhail. 

In the 2015-16 school year, the Online School Partnership overall offered 687 group lessons (serving 
1,031 students), 205 sectionals (328 students), and 769 individual lessons (106 students). Total student 
counts may contain duplicates across the different kinds of lessons, as a student may have participated 
in multiple kinds of lessons (i.e. a student may have been involved in group lessons and also have 
received individual lessons). 
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™ Sectionals: In these lessons, MacPhail Teaching Artists work to boost the skills of 
specific sections of choir, band, or jazz bands. They work on repertoire the participating 
students are playing in large group. These students work on development of sound, 
rhythmic precision, general musical understanding, and performance skills. 


™ Individual lessons: These lessons offer one-on-one instruction with an instrument or 
vocal specialist to work on general tone and technique. Teaching Artists may also 
provide support as a student prepares for all-state band or choir and contest preparation. 
Four of the five schools involved in this evaluation had students who took individual 
lessons. Thirty-two students were involved in a total of 597 individual lessons across 
the four schools during the school year. 


Fifty-three students took lessons through the Online School Partnership at these schools: 
9 from Kerkhoven-Murdock-Sunburg High School, 15 from Willmar High School, seven 
from MACCRAY High School, 11 from Rush City High School, and 11 from Lincoln 
High School. Schools had between 12 to 297 sessions throughout the 2015-16 school 
year, and 722 sessions across all of these schools. The schools completed 90 percent of 
these sessions." 


3 Schools had to cancel classes for a variety of reasons, including snow days in which school was 


cancelled and other unexpected schedule changes. 
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Evaluation 


Wilder Research worked with the Online School Partnership to develop an evaluation 
design for the program. The evaluation includes implementation and outcome components 
and gathers qualitative and quantitative data to answer the following research questions. 
(The Online School Partnership developed a logic model and these research questions 
reflect the activities and outcomes of the program. See Appendix). 


Implementation evaluation key questions 


™@ How well is the Online School Partnership program being implemented? What are 
aspects of the program, if any, that need to be adjusted over time? 


m How well does the collaboration between MacPhail and the schools function, and 
how can it be strengthened? 


Outcome evaluation key questions 


=m To what extent does student participation in the Online School Partnership program 
affect their interest, motivation, knowledge, and skills in music? 


@ To what extent does student participation in the Online School Partnership program 
affect their attitudes about school (school motivation and engagement) and student 
non-academic skills? 


m To what extent does music educator participation in the Online School Partnership 
program affect their growth as an educator, their job satisfaction, and ability to handle 
many aspects of their work assignment? 


Methods 


MacPhail and Wilder Research staff worked collaboratively to complete this evaluation. Both 
MacPhail and Wilder staff completed data collection responsibilities, and Wilder Research 
staff completed data entry, data analysis, and reporting. 


The following are the data collection instruments used in the evaluation. 


™ Teaching Artist survey: In spring 2016, Wilder conducted a web survey with the 
MacPhail Teaching Artists. This survey asked about the overall vision and goals of 
the program, how collaboration with the partner schools has been going and how to 
improve the collaboration, successes of the Online School Partnership, and areas for 
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improvement. Eight of 13 Teaching Artists completed the survey, for a response rate 
of 62 percent. 


Interviews with school music educators: Wilder conducted a phone interview with the 
school music educators involved with the Online School Partnership program. In these 
interviews, educators were asked about the value and successes of the program, 
collaboration with MacPhail, and suggestions for program improvement. Of the five 
educators identified, four were interviewed for this evaluation. Due to a technical 
error, the survey intended for Teaching Artists involved with the program was also 
sent to music educators. The fifth music educator completed the survey. Because 
some of the questions in the Teaching Artist survey are the same as the music educator 
survey, their responses to those questions are included. 


Student survey: This survey was developed to measure student satisfaction with the 
Online School Partnership program, perceptions of the lessons provided, and interest 
in music education and school generally. In spring 2016, 51 sixth- through twelfth- 
graders completed a survey, representing 96 percent of the 53 students in the program. 
Fifty-one percent took Online School Partnership lessons as a class, and 49 percent of 
these students received individual lessons. None of these students were involved in 
sectional lessons. 


Music skills assessments: At the beginning and end of students’ time with the Online 
School Partnership, the MacPhail Teaching Artist they worked with completed skills 
assessments, looking at a variety of components of students’ performance abilities 
(tone quality, intonation, rhythm, etc.). Music skill assessments were done before and 
after Online School Partnership programming for 33 of 53 students (62%) involved in 
the Online School Partnership: 8 from Kerkhoven-Murdock-Sunburg (89%), 9 from 
Willmar (60%), 5 from MACCRAY (71%), and 11 from Lincoln (100%). No skill 
assessments were done for students at Rush City. 


Lesson progress report: A progress report was designed to help assess students’ 
musical and behavioral or non-academic skills. In spring 2016, Teaching Artists 
completed 26 progress reports, made up of reports for nine Kerkhoven-Murdock- 
Sunburg students (100%), nine Willmar students (60%), six MACCRAY students 
(86%), and two Lincoln students (18%). No progress reports were done for students 
at Rush City. This represents 49 percent of students participating in Online School 
Partnership program. 
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Results 


This section describes the implementation and outcome results. Results are reported across 
the schools, and separately when appropriate. Results should be interpreted with caution 
given the small sample size. 


Implementation 
Successes 


Programming 


When asked about the biggest successes of the Online School Partnership program in the 
interview, two of the four school music educators pointed to the higher quality of instruction. 
One music educator spoke about how personable the Teaching Artists are, which has 
increased their students’ level of attention in sessions. Another music educator spoke 
about how the Online School Partnership has greatly increased students’ level of interest 
in music. Another music educator stated that the external funding of the program contributed 
to the success of the program. 


Students indicated that they were satisfied with the Online School Partnership program. 
Students were asked to what degree they agree with the statements “I like coming to the 
online sessions” and “I would recommend these sessions to other students.” Eighty-six 
percent of students stated that they “agree” or “strongly agree” that they like coming to 
the online sessions, and 90 percent would recommend the sessions to other students (Figure 1). 


In terms of satisfaction with the technology used for the online sessions, 87 percent agreed 
or strongly agreed with the statement “the technology used for these sessions made it easy 
to learn” (Figure 2). From MacPhail’s perspective, the technology makes collaborating with 
instrument and genre specialists accessible and readily available no matter where students 
and music educators are located. By offering sessions to take place via the Telepresence 
and Skype (two kinds of teleconferencing software), MacPhail works to ensure there is the 
opportunity for simultaneous sessions and increased accessibility to lessons. The degree to 
which students strongly agreed with this statement varied by school, with students at Lincoln 
more likely to strongly agree with this than their peers at other schools (61% strongly 
agreed at Lincoln, as compared to 31% of students at all schools). 


Students were also positive about the Teaching Artist who led their online sessions; 90 

percent of students said they liked their Teaching Artist (Figure 3). Students at Willmar 
and Lincoln were particularly positive, with 80 percent and 100 percent of students strongly 
agreeing that they liked their Teaching Artist, respectively. Students also said their Teaching 
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Artist used class time well (92% agree or strongly agree) and set clear expectations for 
practicing and learning at home (84% agree or strongly agree). 


When asked what they liked most about the online sessions, students most frequently 
noted that they enjoyed that they gained new skills, techniques, or musical knowledge 
(48%; Figure 4). 


Collaboration 


In the interview, the music educators were asked to describe the collaborative efforts between 
their school and MacPhail. Four music educators offered praise about MacPhail’s flexibility 
and attentiveness to the needs of their school. A couple of music educators were particularly 
positive about MacPhail’s attention to what music instructors wanted them to focus on in 
lessons. They stated that MacPhail used their instruction time and resources well to ensure 
that those focus areas were paid attention to. One teacher spoke about how respectful 
MacPhail and the Teaching Artist were; the Teaching Artist was able to make suggestions 
as to how the music instructor could improve their lessons, but did so in a way that was 
supportive. 


Opportunities for improvement 


Programming 


The music educators did not offer suggestions for programming improvements. Rather, their 
suggestions related to logistical aspects of the program. Two music educators suggested 
that the program offer more support in keeping track of hours used by the schools, and three 
noted concerns with the payment structure of the program. These music educators were 
concerned that schools would not be able to participate due to the cost. Another couple of 
music educators noted that the state needs faster internet speeds in rural areas, and one 
suggested that MacPhail lobby the state to prioritize this issue. One teacher requested that 
she be able to have a male Teaching Artist teach her male choir students, but said that 
wasn’t available at present. 


Collaboration 


No common themes arose around recommendations to improve collaboration from the music 
educator interviews. A couple of music educators noted that technical issues and delay were 
somewhat of a problem, but both acknowledged that this issue is not in MacPhail’s hands. 
Similarly, one teacher noted that he wished he had a larger, established space to hold lessons, 
but acknowledged that MacPhail was not able to do anything about that. Aside from this, 
one teacher requested there be more communication about policies surrounding missed 
classes, one requested more flexibility in scheduling and rescheduling, and one expressed 
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concern about schools having a difficult time finding funding for the Online School 
Partnership. One teacher stated that they had no suggestions for collaboration and that they 
only had positive things to say. 


Outcomes 


Benefits to students in music 
Students gain musical skills and knowledge 


According to the MacPhail Teaching Artists, almost all students at Kerkhoven-Murdock- 
Sunburg, Willmar, and MACCRAY attended lessons regularly, with 95 percent of students 
in the combined schools attending 80 percent or more of the offered lessons (Figure 5). 
Students also come prepared to lessons. Teaching Artists reported that students at the partner 
schools either “always” (21%) or “most of the time” (46%) prepared assigned music in home 
practice (Figure 6). Ninety-seven percent of students were reported to have shown consistent 
musical progress and growth “always” or “most of the time.” 


Teaching Artists at Kerkhoven-Murdock-Sunburg and Willmar reported on the amount of 
time students lead an activity in class or help plan activities for class. Students were very 
engaged in these classroom activities, with 100 percent of students participating in both 
leading an activity and helping to plan activities for class (Figure 6). Teaching Artists 
only rated students on their frequency of helping to plan and leading activities in class if 
they deemed it relevant for the student; eighteen students were rated on how frequently 
they helped plan activities for class, and three students were rated on how frequently they 
lead an activity in class. 


When students were asked to how much they agree with the statement “I learned new 
musical skills that helped me in band/choir,” students were generally positive; 92 percent 
either “agree” or “strongly agree” with the statement (Figure 7). Fewer, but still a high 
percentage of students (73%) at MACCRAY strongly agreed or agreed with this statement. 


Teaching Artists assessed students’ musical skills at the beginning and end of their time 
with the Online School Partnership. Students were assessed in a variety of areas: tone quality; 
intonation; rhythm; balance blend; technique; interpretation/musicianship; diction, bowing, 
articulation, or execution (depending on the student’s instrument); and “other performance 
factors” (Figures 8-15). Overall, students showed moderate growth in most skills. Students 
grew most in interpretation/musicianship, with an improvement of .6 points to a mean 
score of 3.5 at the end of their lessons (where 1=poor, 2=fair, 3=good, 4=excellent, and 
5=superior; Figure 13). Students scored highest in the “other performance factors” category, 
which includes general conduct, mannerisms, poise, and posture (Figure 15). The mean 
score for all students was 3.8 at the end of their lessons. 
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In interviews, all music educators indicated that the Online School Partnership has increased 
student skill and knowledge. One mentioned that it provides a great opportunity to apply 
musical theory learned in a classroom setting. Another mentioned that their students are 
achieving higher scores at competitions. 


Students show interest and motivation in music 


On student demonstration of an eagerness to learn, Teaching Artists gave ratings of “always” 
or “most of the time” to 92 percent of students (Figure 16). Both Lincoln students, seven 
Kerhoven-Murdock-Sunburg students, three MACCRAY students, and four Willmar 
students were said to “always” demonstrate an eagerness to learn. 


Eighty-nine percent of students reported that they “agree” or “strongly agree” that coming 
to the online sessions increased their appreciation of music (Figure 17). Students were 
interested in continuing to develop their musical skills and are more motivated to do well 
on their instruments (92% each). Looking to the future, almost half of the students (46%) 
stated that they are interested in pursuing a college degree in music. 


While students were very positive generally, there was some variance among the different 
schools. Students at Lincoln were more likely to “strongly agree” that coming to the online 
sessions increased their appreciation of music (83%, compared to 51% overall; Figure 17). 
Students at Lincoln were also more likely to strongly agree that they are interested in 
continuing to develop their musical skills (89%, compared to 47 percent overall). 


To assess their motivation to continue developing their musical skills, students were asked 
if they wanted to continue in the MacPhail program the following year. Eighty-two percent 
of all students wanted to continue in the program (Figure 17). Forty-seven percent of the 
students responded that they “strongly agree” that they want to continue in the program, 
and an additional 35 percent said they “agree” that they want to continue. Nine percent of 
students “disagree” and another nine percent of students “strongly disagree” that they would 
like to continue. A higher percentage of Lincoln students (65%) said they “strongly agree” 
that they would like to continue, compared to students overall (47%). MACCRAY students 
were least likely to say they would like to continue in the program (45% saying they 
“disagree” or “strongly disagree”’). Again, the results should be interpreted with caution 
due to the small number of students in each school. 
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Benefits to students in school 
Students have positive attitudes toward school 


Students reported that they work hard in school, with 98 percent of students at all schools 
either agreeing or strongly agreeing that they do so (Figure 18). 


In interviews, music educators were hesitant to say that the Online School Partnership was 
improving students’ general attitudes about school. Two music educators spoke about 
how the Online School Partnership is just another component of students’ days or how 
the students who opt in to the lessons want to be there in the first place. The other two 
music educators indicated that there was some benefit to students’ attitudes. Both stated 
that these lessons provide students who are unsure of where they fit in at school an avenue 
for self-discovery and confidence. By increasing their knowledge and skills in music, 
they are able to find a niche for themselves, and by doing so, are more comfortable and 
positive about their schooling experience. 


Students show positive non-academic skills 


All students participating in the Online School Partnership were said to bring all their 
required materials either “always” (73%) or “most of the time” (27%) to their sessions 
(Figure 19). Students were also cooperative in a group setting; Teaching Artists rated 
eight students on this measure, and all students were said to “always” be cooperative. 
Additionally, students followed directions “always” or “most of the time” in lessons 
(100%) and persisted on tasks even when experiencing difficulty (92%). 


Students at the partner schools were less likely to make an effort to try out possible solutions 
when encountering a problem “always” or “‘most of the time” (54%; Figure 19) and to make 
an effort to begin an assignment before asking for assistance “always” or “most of the 
time” (77%). 


Benefits to music educators and schools 


Music educators grow as teachers, and have increased resources and capacity to 
grow their music programs 


All four music educators were very positive about their own growth and learning due to 
participation in the program. When asked to what extent the Online School Partnership 
benefited them in interviews, three of the music educators spoke about the benefits of 
being able to observe the Teaching Artists give lessons or bounce ideas off them. A couple 
of music educators mentioned being able to go to the Teaching Artist with any questions, 
which was particularly helpful due to the lack of peers at their school or in their general 
geographical region. One teacher was particularly vocal about the increased capacity of their 
music program with the addition of the Online School Partnership, stating that having a 
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Teaching Artist working with their class allows for specialized knowledge and tailored 
teaching that would otherwise be difficult with their schedule and other job demands. 


All educators were also positive about the impact the Online School Partnership is having 
on their school. A couple of educators mentioned that more students were getting involved 
in music as they heard about the program. One teacher mentioned that there were now 
twice as many students enrolled in music classes than the year before, and attributed this 
growth to the growing awareness of the Online School Partnership. 


Two music educators mentioned that this program was greatly increasing the capacity of 
their music departments to provide instruction to a greater number of students. While a 
teacher may be in session with a regular band class, for example, an ensemble of students 
can receive more tailored lessons from a Teaching Artist in another room. 


Issues to consider 


The Online School Partnership data indicates that participants are doing well in the program. 
Students generally enjoy their classes and are actively engaged and progressing in their 
music education. Similarly, music educators are positive about their satisfaction with the 
program and generally feel it is a helpful resource for their students and themselves. 


Based on the results available to date, the following are a few points to consider in future 
planning for the program. MacPhail leaders and staff may have already made adjustments 
in some of these areas. 


™ MacPhail program staff should continue in being flexible and attentive to the partner 
schools’ needs. 


@ MacPhail program staff could create tools to help partner schools more easily track 
program logistics, including scheduled lesson times and how many lessons students have 
completed. 


™ MacPhail program staff could alleviate cost concerns for partner schools by providing 
a greater amount of funding, consistent funding over the course of a few years, or by 
helping schools find and secure funding. 


™ MacPhail should encourage Teaching Artists and partner schools to collect more 
surveys, progress reports, and skill assessments to better evaluate the program. 


™ MacPhail program staff could advocate at the state level for greater access to high- 
speed internet in rural areas. 
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Figures 


Implementation 


1. Students’ general satisfaction with the Online School Partnership 


Strongly Strongly 
disagree Disagree Agree agree 
School 1 2 3 4 Mean 
| like coming to Kerkhoven-Murdock- 4 0% 0% 100% 0% 3.00 
the online Sunburg High School 
cic Willmar High School 10 0% 10% 70% 20% 3.10 
MACCRAY High School 11 27% 27% 36% 9% 2.27 
Rush City High School 7 0% 0% 100% 0% 3.00 
Lincoln High School 18 0% 0% 28% 72% 3.72 
All schools 50 6% 8% 54% 32% 3.12 
| would Kerkhoven-Murdock- 4 0% 0% 50% 50% 3.50 
recommend Sunburg High School 
these sessions to 6 e ° ° 
athorstudents. Willmar High School 9 0% 0% 56% 44% 3.44 
MACCRAY High School 10 30% 20% 40% 10% 2.30 
Rush City High School 7 0% 0% 71% 29% 3.29 
Lincoln High School 18 0% 0% 11% 89% 3.89 
All schools 48 6% 4% 38% 52% 3.35 
2. Students’ satisfaction with technology used 
Strongly Strongly 
disagree Disagree Agree agree 
School 1 2 3 4 Mean 
The technology Kerkhoven-Murdock- 4 0% 0% 100% 0% 3.00 
used for these Sunburg High School 
sessions made it 5 4 6 : 
easy to learn. Willmar High School 10 0% 20% 60% 20% 3.00 
MACCRAY High School 9 11% 22% 56% 11% 2.67 
Rush City High School 7 0% 0% 86% 14% 3.14 
Lincoln High School 18 0% 6% 33% 61% 3.56 
All schools 48 2% 10% 56% 31% 3.17 
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3. Students’ report of instructor 


Strongly Strongly 
disagree Disagree Agree agree 
School N 1 2 3 4 Mean 
| liked the Kerkhoven-Murdock- 4 0% 0% 75% 25% 3.25 
Teaching Artist Sunburg High School 
ora the Willmar High School 10 0% 0% 20% 80% 3.80 
MACCRAY High School 10 30% 20% 40% 10% 2.30 
Rush City High School 8 0% 0% 75% 25% 3.25 
Lincoln High School 18 0% 0% 0% 100% 4.00 
All schools 50 6% 4% 30% 60% 3.44 
The Teaching Kerkhoven-Murdock- 4 0% 0% 50% 50% 3.50 
Artist used class = Sunburg High School 
time well. Willmar High School 10 0% 0% 50% 50% 3.50 
MACCRAY High School 9 11% 11% 33% 44% 3.11 
Rush City High School 8 0% 13% 50% 38% 3.25 
Lincoln High School 18 0% 6% 11% 83% 3.78 
All schools 49 2% 6% 33% 59% 3.49 
The Teaching Kerkhoven-Murdock- 4 0% 25% 25% 50% 3.25 
Artist set clear Sunburg High School 
expectations for = willmar High School 9 0% 0% 44% 55% 3.56 
practicing and . 
learning at home. MACCRAY High School 11 27% 18% 27% 27% 2.55 
Rush City High School 8 0% 13% 50% 38% 3.25 
Lincoln High School 17 0% 6% 41% 53% 3.47 
All schools 49 6% 10% 39% 45% 3.22 


Note. Totals may not equal 100 due to rounding 
Note. Percentages have been added for consistency, but it is important to note the small sample size. 


4. Student satisfaction with Online School Partnerships classes 
What did you like most about this class? N=27 N Percent 
New skills/technique/knowledge 13 48% 
Time with the Teaching Artist 4 15% 
Class activities 3 11% 
Other 4 15% 
General negativity 4 15% 


Note. Students may have noted more than one thing they liked most about the class, resulting in a total 
>100%. Percentages given reflect the number of times a response was given divided by the total number 


of responses. 


Note. “Other” responses include general praise, and positive notes about the song selection. 


MacPhail Online School Partnership 


2015-16 Evaluation Results 


12 


Wilder Research, November 2016 


Outcomes 


5. MacPhail Teaching Artists report of student attendance rates 


Kerkhoven-Murdock- MACCRAY 
Sunburg High School _— Willmar High School High School All schools 
Percentage of (N=9) (N=9) (N=6) (N=26) 
lessons attended N % N % N % N % 
100% 6 75% 5 56% 1 25% 12 57% 
90-99% 1 13% 3 33% 0 0% 4 19% 
80-89% 1 13% 0 0% 3 75% 4 19% 
Less than 80% 0 0% 1 11% 0 0% 1 5% 
Average 97% 95% 87% 94% 
Note. Totals may not equal 100 due to rounding 
Note. This information was not available for Rush City or Lincoln High School. 
6. Progress report results: increased student musical skill and knowledge 
Most of 
Never Sometimes the time Always 

Behavior School 1 2 3 4 Mean 
The student Kerkhoven-Murdock- 0% 25% 38% 38% 3.1 
prepares assigned Sunburg High School 
ae One Willmar High School 8 0% 38% 50% 13% 2.8 
practice 

MACCRAY High School 6 0% 50% 33% 17% 2.7 

Lincoln High School 2 0% 0% 100% 0% 3.0 

All schools 24 0% 33% 46% 21% 2.9 
The student has Kerkhoven-Murdock- 9 0% 0% 44% 56% 3.6 
shown consistent Sunburg High School 
musical progress witimar High School 9 0% 0% 89% 11% 3.1 
and growth 

MACCRAY High School 6 0% 17% 50% 33% 3.2 

Lincoln High School 0% 0% 0% 100% 4.0 

All schools 26 0% 4% 58% 39% 3.3 
Student leads an Kerkhoven-Murdock- 2 0% 0% 0% 100% 4.0 
activity in class? Sunburg High School 

Willmar High School 0% 0% 0% 100% 4.0 

All schools 0% 0% 0% 100% 4.0 
Student helps Kerkhoven-Murdock- 0% 0% 0% 100% 4.0 
plan activities Sunburg High School 
ror easas Willmar High School 9 0% 0% 0% 100% 4.0 

All schools 26 0% 0% 0% 100% 4.0 
Teaching Artists only rated the students this was relevant for. 
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7. Students’ report of musical skills and knowledge 


Strongly Strongly 
disagree Disagree Agree agree 
School 1 2 3 4 Mean 
| learned new Kerkhoven-Murdock- 4 25% 0% 0% 75% 3.25 
musical skills that Sunburg High School 
Snead Willmar High School 10 0% 0% 30% 70% 3.70 
MACCRAY High School 11 18% 9% 46% 27% 2.82 
Rush City High School 8 0% 0% 50% 50% 3.50 
Lincoln High School 17 0% 0% 24% 77% 3.76 
All schools 50 6% 2% 32% 60% 3.46 
8. Tone quality before and after online sessions 
Superior Excellent Good Fair Poor 
School N 5 4 3 2 1 Mean Change 
Kerkhoven-Murdock- Pre 8 0% 26% 63% 13% 0% 3.4 
Sunbuig ienenee” | Pesel. 6 0% 88% 13% 0% 0% 3.9 ere 
Willmar High School Pre 9 0% 44% 33% 22% 0% 3.2 sie 
Post 9 11% 44% 44% 0% 0% ot 
MACCRAY High School Pre 5 0% 0% 60% 40% 0% 2.6 oe 
Post 5 0% 20% 80% 0% 0% 3.2 
Lincoln High School Pre 11 0% 18% 82% 0% 0% 3.2 vr 
Post 11 0% 18% 82% 0% 0% 3.2 
All schools Pre 33 0% 24% 61% 15% 0% 3.1 
Post 33 3% 42% 55% 0% 0% 3.5 ea 
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9. Intonation before and after online sessions 


Superior Excellent Good Fair Poor 
School N 5 4 3 2 1 Mean Change 
Kerkhoven-Murdock- Pre 8 0% 50% 38% 13% 0% 3.3 
SUNEMIGTIGN ENO! Peat |. 6 13% 63% 25% 0% 0% 3.9 a 
Willmar High School Pre 9 0% 44% 33% 22% 0% 3.2 te5 
Post 9 22% 44% 33% 0% 0% 3.9 
MACCRAY High School _ Pre 5 0% 20% 40% 40% 0% 2.8 ie 
Post 5 0% 20% 60% 20% 0% 3.0 
Lincoln High School Pre 11 0% 18% 82% 0% 0% 3.1 ee 
Post 11 0% 0% 100% 0% 0% 3.0 
All schools Pre 33 0% 33% 52% 15% 0% 3.2 
Post 33 9% 30% 58% 3% 0% 3.5 — 
10. Rhythm before and after online sessions 
Superior Excellent Good Fair Poor 
School N 5 4 3 2 1 Mean Change 
Kerkhoven-Murdock- Pre 9 0% 56% 33% 11% 0% 3.4 
SUNBUIOIENSENee: | Bagh |. 6 22% 78% 0% 0% 0% 4.2 aed 
Willmar High School Pre 9 0% 44% 44% 11% 0% 3.3 sine 
Post 9 22% 67% 11% 0% 0% 4.1 
MACCRAY High School Pre 6 0% 17% 33% 17% 33% 2.3 ape 
Post 6 0% 33% 17% 50% 0% 2.8 
Lincoln High School Pre 11 0% 9% 91% 0% 0% 3.1 aa 
Post 11 0% 9% 82% 9% 0% 3.0 
All schools Pre 35 0% 31% 54% 9% 6% 3.1 
Post 35 11% 46% 31% 11% 0% 3.6 isk 
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11. Balance blend before and after online sessions 


Superior Excellent Good Fair Poor 
School N 5 4 3 2 1 Mean Change 
Kerkhoven-Murdock- Pre 3 33% 67% 0% 0% 0% 4.3 
SUNMOTICMEENOe! Pest |. 3 0% 100% 0% 0% 0% 4.0 — 
Willmar High School Pre 6 0% 67% 33% 0% 0% 3.7 oe 
Post 6 17% 50% 33% 0% 0% 3.8 
MACCRAY High School Pre 3 33% 0% 0% 33% 33% 2.7 
Post 3 0% 33% 0% 67% 0% 2.7 sad 
Lincoln High School Pre 11 0% 9% 82% 9% 0% 3.0 ve 
Post 11 0% 9% 91% 0% 0% 3.1 
All schools Pre 23 9% 30% 48% 9% 4% 3.3 ae 
Post 23 4% 35% 52% 9% 0% 3.3 
12. Technique before and after online sessions 
Superior Excellent Good Fair Poor 
School N 5 4 3 2 1 Mean Change 
Kerkhoven-Murdock- Pre 9 0% 44% 33% 22% 0% 3.2 
SUNBUOTIOSENOC! |Past .°%9 11% 67% 11% 11% 0% 3.8 st 
Willmar High School Pre 9 0% 11% 78% 11% 0% 3.0 re 
Post 9 0% 44% 56% 0% 0% 3.4 
MACCRAY High School _ Pre 6 0% 0% 67% 17% 17% 2.5 aa 
Post 6 0% 17% 50% 33% 0% 2.8 
Lincoln High School Pre 11 0% 9% 91% 0% 0% 3.1 a4 
Post 11 0% 18% 82% 0% 0% 3.2 
All schools Pre 35 0% 17% 69% 11% 3% 3.0 
Post 35 3% 40% 51% 6% 0% 3.4 
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13. Interpretation/Musicianship before and after online sessions 


Superior Excellent Good Fair Poor 
School N 5 4 3 2 1 Mean Change 
Kerkhoven-Murdock- Pre 9 0% 22% 56% 22% 0% 3.0 
SUMMON ENOO! Peat |, 9 0% 100% 0% 0% 0% 4.0 id 
Willmar High School Pre 9 0% 22% 44% 33% 0% 2.9 
Post 9 11% 44% 44% 0% 0% 7 sik 
MACCRAY High School _ Pre 6 0% 17% 33% 17% 33% 2.3 a7 
Post 6 0% 50% 17% 17% 17% 3.0 
Lincoln High School Pre 11 0% 9% 91% 0% 0% 3.1 
Post 11 0% 9% 91% 0% 0% 3.1 — 
All schools Pre 35 0% 17% 60% 17% 6% 2.9 cae 
Post 35 3% 49% 43% 3% 3% 3.5 
14. Diction/Bowing/Articulation/Execution before and after online sessions 
Superior Excellent Good Fair Poor 
School N 5 4 3 2 1 Mean Change 
Kerkhoven-Murdock- Pre 9 0% 56% 33% 11% 0% 3.4 
SUNBUO TION SENOC!: | Past4.°%9 0% 78% 22% 0% 0% 3.8 pia 
Willmar High School Pre 9 0% 44% 56% 0% 0% 3.4 outs 
Post 9 0% 67% 33% 0% 0% 3.7 
MACCRAY High School Pre 6 17% 0% 33% 33% 17% 27 aoe 
Post 6 0% 33% 33% 33% 0% 3.0 
Lincoln High School Pre 2 0% 50% 50% 0% 0% 3.5 ane 
Post 2 0% 100% 0% 0% 0% 4.0 
All schools Pre 26 4% 39% 42% 12% 4% 3.3 
Post 26 0% 65% 27% 8% 0% 3.6 _ 
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15. Other performance factors before and after online sessions 


Superior Excellent Good Fair Poor 

School N 5 4 3 2 1 Mean Change 
Kerkhoven-Murdock- Pre 9 11% 33% 44% 11% 0% 3.4 
SUNBUID IGEN! peat |) 20 22% 67% 11% 0% 0% 44 cal 
Willmar High School Pre 9 22% 56% 22% 0% 0% 4.0 65 

Post 9 22% 78% 0% 0% 0% 4.2 
MACCRAY High School _ Pre 6 17% 17% 67% 0% 0% 3.5 

Post 6 17% 50% 33% 0% 0% 3.8 sal 
Lincoln High School Pre 11 0% 18% 82% 0% 0% 3.2 

Post 11 0% 18% 82% 0% 0% 3.2 = 
All schools Pre 35 11% 31% 54% 3% 0% 3.5 

Post 35 14% 51% 34% 0% 0% 3.8 oe 


Note. One teacher indicated that a student was between “Good” and “Excellent” in tone quality. This response has been rounded up into the 


“Excellent” category. 


Note. Two music educators indicated that a student was between “Good” and “Excellent” in intonation. This response has been rounded up into the 


“Excellent” category. 


16. Progress report results: student interest and motivation 


Most of 
Always the time | Sometimes Never 
Behavior School N 4 3 2 1 Mean 
The student Kerkhoven-Murdock- 9 78% 22% 0% 0% 3.8 
demonstrates Sunburg High School 
he aaa i Willmar High School 9 44% 33% 22% 0% 3.2 
MACCRAY High School 6 50% 50% 0% 0% 3.5 
Lincoln High School 2 100% 0% 0% 0% 4.0 
All schools 26 62% 30% 8% 0% 3.5 
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17. Students’ report of musical interest 


Strongly Strongly 
disagree Disagree Agree agree 
School 1 2 3 4 Mean 
Coming to the Kerkhoven-Murdock- 4 0% 25% 50% 25% 3.00 
online sessions Sunburg High School 
increased my ; 5 - 7 5 
appreciation of Willmar High School 10 0% 0% 60% 40% 3.40 
music. MACCRAY High School 11 27% 9% 46% 18% 2.55 
Rush City High School 4 0% 0% 50% 50% 3.50 
Lincoln High School 18 0% 0% 17% 83% 3.83 
All schools 47 6% 4% 38% 51% 3.34 
| am interested in © Kerkhoven-Murdock- 3 0% 0% 100% 0% 3.00 
continuing with Sunburg High School 
ae Willmar High School 7 0% 29% 14% 57% 3.29 
Partnership MACCRAY High School = 11 36% 9% 27% 27% 2.45 
Program 
: Rush City High School 5 0% 20% 40% 40% 3.20 
Lincoln High School 17 0% 0% 35% 65% 3.65 
All schools 43 9% 9% 35% 47% 3.19 
lam interested in Kerkhoven-Murdock- 4 0% 0% 100% 0% 3.00 
continuing to Sunburg High School 
develop my . 2 ‘ . 2 
musical sialls: Willmar High School 10 0% 0% 30% 70% 3.70 
MACCRAY High School 10 20% 20% 30% 30% 2.70 
Rush City High School 8 0% 0% 63% 38% 3.38 
Lincoln High School 18 0% 0% 11% 89% 3.89 
All schools 50 4% 4% 34% 58% 3.46 
| am interested in © Kerkhoven-Murdock- 1 0% 100% 0% 0% 2.00 
pursuing a Sunburg High School 
college degree in . 6 4 . 2 
witisin after Willmar High School 8 13% 38% 13% 38% 2.75 
graduating from =MACCRAY High School 9 56% 33% 11% 0% 1.56 
high school. a 
Rush City High School 7 0% 43% 29% 29% 2.86 
Lincoln High School 12 17% 17% 25% 42% 2.92 
All schools 37 22% 32% 19% 27% 2.51 
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17. Students’ report of musical interest (continued) 


Strongly Strongly 
disagree Disagree Agree agree 
School N 1 2 3 4 Mean 
lam more Kerkhoven-Murdock- 4 0% 25% 75% 0% 2.75 
motivated to do Sunburg High School 
well on my 5 - 7 7 
jnstrdnient Willmar High School 10 0% 0% 60% 40% 3.40 
MACCRAY High School 9 11% 22% 44% 22% 2.78 
Rush City High School 8 0% 0% 75% 25% 3.25 
Lincoln High School 18 0% 0% 44% 56% 3.56 
All schools 49 2% 6% 55% 37% 3.27 
18. Students’ report of work in school 
Strongly Strongly 
disagree Disagree Agree agree 
School N 1 2 3 4 Average 
| work hard in Kerkhoven-Murdock- 4 0% 0% 25% 75% 3.75 
school. Sunburg High School 
Willmar High School 10 0% 0% 40% 60% 3.60 
MACCRAY High School 11 9% 0% 46% 46% 3.27 
Rush City High School 8 0% 0% 88% 13% 3.30 
Lincoln High School 18 0% 0% 33% 67% 3.67 
All schools 51 2% 0% 45% 53% 3.49 
Note. Table draws on information from the Student Survey. 
Note. Totals may not equal 100 due to rounding 
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19. Progress report results: student growth in non-academic skills 


Most of 
Always the time Sometimes Never 
Behavior School 4 3 2 1 Mean 
The student Kerkhoven-Murdock- 9 89% 11% 0% 0% 3.9 
brings all Sunburg High School 
required é ‘ 5 é 
materials Willmar High School 9 67% 33% 0% 0% 3.7 
(method book, MACCRAY High School 83% 17% 0% 0% 3.8 
instrument, etc.) i" . . P 
to the group Lincoln High School 2 0% 100% 0% 0% 3.0 
lesson? All schools 26 73% 27% 0% 0% 3.7 
The student is Kerkhoven-Murdock- 3 100% 0% 0% 0% 4.0 
cooperative ina Sunburg High School 
BOPP esting Willmar High School 100% 0% 0% 0% 4.0 
MACCRAY High School 100% 0% 0% 0% 4.0 
Lincoln High School : . : : - ‘i 
All schools 8 100% 0% 0% 0% 4.0 
The student Kerkhoven-Murdock- 100% 0% 0% 0% 4.0 
follows directions © Sunburg High School 
THE HeSson Willmar High School 100% 0% 0% 0% 4.0 
MACCRAY High School 6 83% 17% 0% 0% 3.8 
Lincoln High School 2 100% 0% 0% 0% 4.0 
All schools 26 96% 4% 0% 0% 4.0 
Student persists Kerkhoven-Murdock- 9 100% 0% 0% 0% 4.0 
on tasks even Sunburg High School 
when i i 0, fe} 0, 0, 
experiencing Willmar High School 22% 56% 22% 0% 3.0 
difficulty MACCRAY High School 67% 33% 0% 0% 3.7 
Lincoln High School 2 100% 0% 0% 0% 4.0 
All schools 26 65% 27% 8% 0% 3.6 
Whena problem = Kerkhoven-Murdock- 8 38% 38% 25% 0% 3.4 
is encountered, Sunburg High School 
student makes : 7 a - e 
own effort to try Willmar High School 22% 22% 56% 0% 2.7 
out possible MACCRAY High School 20% 0% 80% 0% 2.4 
solutions ; ; 
Lincoln High School 2 0% 100% 0% 0% 3.0 
All schools 24 25% 29% 46% 0% 2.8 


Note. Totals may not equal 100 due to rounding 


Note. Rush City did not complete any lesson progress reports. 


a Percentages have been added for consistency, but it is important to note the small sample size. 
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19. Progress report results: student growth in non-academic skills (continued) 


Most of 
Always the time Sometimes Never 

Behavior School 4 2 1 Mean 
Student makes Kerkhoven-Murdock- 7 29% 57% 0% 14% 3.0 
own efforts to Sunburg High School 
begin assignment : ‘ e 5 
before asking for Willmar High School 6 17% 33% 50% 0% 2.7 
assistance MACCRAY High School 2 50% 50% 0% 0% 3.5 

Lincoln High School 2 0% 100% 0% 0% 3.0 

All schools 17 24% 53% 18% 6% 2.9 
Note. Totals may not equal 100 due to rounding 
Note. Rush City did not complete any lesson progress reports. 

MacPhail Online School Partnership 22 Wilder Research, November 2016 


2015-16 Evaluation Results 


Appendix 
Open-ended comments 


A1. Student survey open-ended comments: “What did you like most about this class?” 


Rush City (N=5) 


Singing opportunities (2) 
Singing pop songs. 
Getting to sing. 

Increased musical skill and knowledge (3) 
When we learned new methods. 
Develop my skills. 

Improving with exercises and warm ups. 
MACCRAY (N=10*) 

Increased musical skill and knowledge (3) 
Learning new things on the trombone. 
Learning different rhythms. 

Learning new things. 
Class activities (2) 
Getting outside and marching. 
Online sections. 
Other (6) 
Nothing. 
| don’t know. 
Everything. 
Mehh. 
Leaving. 
Nothing, it didn’t interest me. 
Willmar High School (N=9*) 

Increased musical skill and knowledge (6) 
Learning to transpose music, which really helps me read piano music the right way on the trumpet. 
It helped me improve technically on my instrument, as well as improve my tone and intonation. 


Working to improve the things | was terrible at. 


Note. Responses may have been edited for clarity. 
* Responses might have been broken up and arranged into their respective categories 
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Ai. Student survey open-ended comments: “What did you like most about this class?” 
(continued) 


Willmar High School (N=9*) (continued) 


Increased musical skill and knowledge (6) (continued) 


Learning from a different person who has different ideas to possibly help me learn a concept better | have had 
trouble with before. 


Getting to learn new techniques. 
Learning my solo. 
Support and encouragement (2) 
| was pushed to do a solo for solo ensemble. 


| loved how motivating my teacher was. He truly believed in me. | thought it was impossible for me to make All- 
State band, but with his help and encouragement, | completed my goal. 


Other (1) 
The one-on-one time with the teacher. 
Kerkhoven-Murdock-Sunburg High School (N=4) 
| liked that it wasn’t super strict but not totally laid back as well. It was flexible to me and my learning. 
Getting to learn new techniques. 
Learning more about playing. 


The one-on-one instruction. 


Note. Responses may have been edited for clarity. 
* Responses might have been broken up and arranged into their respective categories 
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A2. Student survey open-ended comments: “What would you improve about this class?” 


Rush City (N=5) 


Drama (less of it) 
How nobody listens; the songs we sing 
Making it more entertaining and organized. 
Less warming up. We don't get to work on the real songs. 
Timing. | think warm ups take a lot longer than they should. 
MACCRAY (N=10*) 
Playing the trombone. (2) 
Nothing. 
The connection, because the computer would go out quite a lot. 
Have not such a delay. 
High notes. (2) 
Be more motivated. 
Meh. 
| don’t know. 
Willmar High School (N=9) 
Nothing, it was all very helpful. 
Not sure. 
Make sure to use every minute of time available in the lesson. 
Nothing. (2) 
Clearer communication on which days we would be meeting. 
Get better Wifi at school. 


Have a list of books made by the teachers sent to our band directors so we are in the same area of learning 
instead of randomly finding stuff to play. 


| don’t know. 

KMS High School (N=4) 
| don’t know. (2) 
Nothing. 


More time. 


* Responses might have been broken up and arranged into their respective categories 
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Logic model 


MacPhail Online School Partnership Logic Model 


Overview: The MacPhail Online School Partnership was established to partner with schools in greater Minnesota to support their music program’s 
goals by providing MacPhail faculty to teach students remotely. By teaching classes remotely, MacPhail aims to increase students’ skills, 
knowledge, and interest in music, and provide the resources for music educators to grow professionally. 


INPUTS ACTIVITIES fo] Uy kere) V T=) 


Teaching artists 


Videoconferencing 
technology 


Teaching artists work with music educators 


to determine curriculum for students 


MacPhail funding 


Tailored lessons are offered 
to students in addition to their regular 
music classes, which include: 
- Group lessons 
- Sectionals 
- Individual lessons 


SHORT TERM 


Student outcomes 


- Students show improved musical skill and 
knowledge'2.3:4 


- Students show increased interest, appreciation, 
and motivation in music® 


- Students show improved attitudes toward school? 
- Students show growth in non-academic skills? 


LONG TERM 


Students continue their 
music education 
throughout high school 
and in college® 


Other funding 


Student musical performances 


Facilities and 
physical space 


Music educator outcomes 
- Music educators grow as teachers‘ 


- Music educators have increased resources and 
capacity to grow their music programs* 


Rural students are more 
likely to have the 
opportunity to perform 
and compete at a high 
levels 


Students 


Music educators observe Teaching Artists 
and learn of new teaching approaches, 
methods, and resources 


MacPhail outcomes 


- MacPhail is able to reach out to students who would 
otherwise have no access to the organization® 


- MacPhail creates partnerships with schools 
throughout the state*® 


Partner schools 
throughout the state are 
better able to offer high- 
quality music education 
to their students® 


Music educators 


Evaluation and documentation 


MacPhail Online School Partnership 
2015-16 Evaluation Results 


‘Measured by music skills assessment 
2 Measured by progress reports 
3 Measured by student survey 
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4Measured by music educator survey 
5 Measured by Teaching Artist survey 
8 Measured by program records 


Survey instruments 


MacPhail 


CENTER FOR 


MUSIC 


Online School Partnerships Progress Report | Completed by Teaching Artist 


Group/Student: Grade: Date: 
MacPhail Instructor: 
School: OQ? KMS OQ? Willmar O° MACCRAY O71 RushCity > Thief River Falls 
Student studies: 
QO? Flute O? Oboe O? Clarinet O)* Saxophone > Horn O® Trumpet 
O® Low Brass OQ? Percussion O?° Voice O* Guitar 
Student’s year in MacPhail program: 
QO? One GQ? Two OU? Three O*% Four O§ Five UO Six 
Attendance: 
Student attended out of classes during the semester. 
Please rate the student’s preparation for and behavior during Most of 
class. Always thetime Sometimes Never 


QO’ Trombone 


Not 
applicable 


1. Student brings all required materials (method book, instrument, 
etc.) to the session. 


2. Student prepares assigned music in home practice. 
3. Student is cooperative in a group setting. 
4. Student follows directions in the lesson. 


5. Student demonstrates eagerness to learn. 


6. Student has shown consistent musical progress and growth. 
Comment: 


7. Student persists on tasks even when experiencing difficulty. 
8. When a problem is encountered, student makes own effort to 
try out possible solutions. 


9. Student makes own efforts to begin assignment before asking 
for assistance. 


Please rate only if applicable to class structure. 


10. Student leads an activity in the class. 


11. Student helps plan activities for the class. 


Additional teacher comments or concerns: 
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MacPhalt | MacPhail Teaching Artist: 


CENTER FOR MUSIC 
Student(s): 

School: 

Director: 

Instrument(s): 


Please place one of these numbers in each box below; then total. 5 = superior; 4 = excellent; 3= good; 2=fair; 1=poor 


| Performance Factors |__| Comments 
Tone Quality 
Clarity, consistency, control, 
focus, warmth 


Intonation 
Accuracy, consistency of pitch 


Rhythm 

Accuracy of note and rest 
values, correctness of meters, 
duration, pulse, steadiness 


Balance Blend 
Accompaniment, awareness 
of ensemble, likeness of 
qualities, matching of 
registers (if applicable) 


Technique (facility/accuracy) 
Attacks, releases, control of 
ranges, mechanical skill 


Interpretation/Musicianship 
Dynamics, emotional 
involvement, artistry, 
phrasing, style, tempo 


Diction — Vocal 
Bowing — Strings 
Articulation — Winds 
Execution - Percussion 


Other Performance Factors 
General conduct, 
mannerisms, poise, posture, 
etc. 


TOTAL POINTS 


Signature of Teaching Artist: 


Date: __ 
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MacPhail Online School Partnership Student Survey 
Spring 2016 


Choir/Band Teacher: Today’s date: 
MacPhail Teaching Artist: Type of session (individual, sectional, or 
class): 


This survey will ask you questions about you and the music sessions you participated in this semester. We 
are interested in your honest thoughts and ideas, so there are no right or wrong answers. 


Your answers are important. Please indicate how much you agree with each of these statements and 
fill out all parts of the survey. 


Strongly Strongly Don’t 
disagree Disagree Agree agree Know 

1. | learned new musical skills that helped me in 

band/choir. LI LP LI Lis LS 
2. | like coming to the online sessions. C1 LC] L]s Ls Ls 

| liked the Teaching Artist who led the class. LI C1 L|s Ls Ls 
4. Coming to the online sessions increased my 

appreciation of music. LU L? Ls Li LI 
5. The Teaching Artist used class time well. Ly} L]2 L]s L}s Ls 
6. The Teaching Artist set clear expectations for 

practicing and learning at home. LI Oe Us Li Ci 
7. lam interested in continuing with the 

MacPhail Online Partnership program. LI By x Lis Ls 
8. |aminterested in continuing to develop my 

musical skills. LI Li LE Li Ls 
9. |am interested in pursuing a college degree in , , - 

music after graduating from high school. U UO U Lt U 
10. |am more motivated to do well on my 

instrument. LU Li Li Li Ls 
11. The technology used for these sessions made 

it easy to learn. LU L Ls Lis Ls 
12. | would recommend these sessions to other 

students. LI By LP EF LF 
13. | work hard in school. L] [2 Ls L}s Ls 

-OVER- 
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15. What did you like most about this class? 


16. What would you improve about this class? 


17. Did you participate in a competition this year? 
L]' No 
Ll? Yes 


If yes, check all that apply: 
L]*  Solo/Ensemble 

L]* Honor Band/Choir 
Lis Al-State 


THANK YOU FOR COMPLETING THE SURVEY! 
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